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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
05/08/2009 has been entered. 

2. This Office Action is response to applicant's Amendment, filed 05/08/2009, in 
which the applicant amends claims 1, 21, 41-47 and 49, adds new claim 52, and 
argues for patentability of the claims over the previously cited prior art. 

Response to Amendment 

3. The applicant's amendments to claims 1, 21, and 41-47 have been considered 
and are accepted. It is noted by the examiner that the current amendments 
substantially change the scope of the limitations of the claims as previously presented. 
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It is noted by the examiner that formal acceptance of the conditions of the claims is not 
an indication of allowability of the claims over the prior art. Appropriate rejections are 
included in this Office Action in the relevant sections below. 

4. The applicant's amendments to claim 49 have been considered, but have not 
been found to be formally acceptable. Appropriate objections and/or rejections are 
included in this Office Action in the relevant sections below. 

5. The applicant's presentation of new claim 52 has been considered, but has not 
been found to be formally acceptable. Appropriate objections and/or rejections are 
included in this Office Action in the relevant sections below. 



Response to Arguments 

6. Applicant's arguments with respect to the previous rejection of claim 49 under 35 
USC §112, First Paragraph, filed 05/08/2009 have been fully considered but they are 
not persuasive. 

In particular, the disclosure of the Specification previously cited to the Examiner 
(Page 15, lines 23-29 and Page 14, line 23 - Page 15, line 18) do in fact provide 
disclosure of the terms "label amplitude" and "label duration." Though the applicant is 
permitted to act as a lexicographer with regards to the present invention, any special 
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meaning given to any term, invented or not, must be present in the Specification as a 
clear and explicit definition in the affirmative. Such a definition is not presented for 
either term within the body of the specification, nor in any of the original disclosure of 
the instant application. Each term is not particularly defined by the applicant and the 
examiner contends that neither term is sufficiently well-known within the art that such a 
definition is unnecessary. The examiner notes that the "labels" of the instant invention 
are defined as static text data, being referred to as "text labels" throughout both the 
instant specification and the claims as currently presented. 

The applicant specifically argues further that "the terms 'amplitude' and 'duration' are 
standard parameters for electronic signals.. .the 'labels' in question are created and stored in an 
electronic format, and therefore may readily possess these attributes (Remarks of 05/07/2009: 
Page 17, paragraph 3)." 

The examiner contends that an appropriate interpretation for each of "label 
amplitude" and "label duration" respectively would be the "amplitude of a label" and the 
"duration of a label." The examiner maintains that it is unclear exactly how the 
otherwise well-known terms of art "amplitude" and "duration" can be applied to static 
text data, which is how the labels are disclosed throughout the instant application. To 
clarify this position, the examiner believes it is analogous to determining the "amplitude" 
and "duration" of a spreadsheet document or other static data. 

The applicant is invited to provide evidence to either definitively and objectively 
discredit this interpretation of the claims based upon the disclosure of the instant 
application or to provide resources from a variety of sources within the prior art that 
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show the common usage of each term. 

In view of the current amendments to claim 49, the examiner believes the 
applicant is selectively interpreting each of "label amplitude parameter" and "label 
duration parameter" to describe the amplitude and duration of the dynamic narration 
data. However, the examiner maintains that not only is such a selective interpretation in 
no way exclusive of the interpretation provided above, the phrasing of the terms in fact 
suggests the examiner's interpretation to one of ordinary skill in the art, wherein the 
"amplitude" and "duration" elements of the terms are modifying the respective "label" 
elements, and not the narrative data the labels are intended to identify. The examiner 
contends that specific disclosure of these elements as "label amplitude" and "label 
duration" further supports this interpretation. 

The examiner now further believes that the applicant had originally intended to 
provide such limitations as "amplitude label parameter" and "duration label parameter," 
which the examiner contends would be a more appropriate recitation to describe what is 
believed to be the intended subject matter. Unfortunately, such a dramatic shift in 
meaning of the claim language, without basis of the specific terms in the specification, 
would comprise new matter being added to the disclosure of the instant application. 

As argued previously, the applicant asserts that "the Specification clearly describes 
how the labels are derived from recorded speech data that similarly has both amplitude and 
duration (Page 18, paragraph 01)." The examiner respectfully disagrees that such a 
teaching is clearly shown in the specification as argued by the applicant, and is unable 
to determine any specific correlation to how the narrative influences the creation of a 
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"label amplitude" to describe the "amplitude" of a text string, or the "label duration" to 
describe the "duration" of a text string. 

However, the examiner contends that the Specification at most makes usage of 
these terms, in particular at the referenced sections of the Specification, but does not 
provide any clear meaning to one of ordinary skill in the art. 

For at least the above reasons, the applicant's arguments with respect to the 
previous rejection of claim 49 under 35 USC 112, First Paragraph are non-persuasive. 

7. Applicant's arguments with respect to previous rejections under 35 USC §1 02(e) 
and 35 USC §1 03(a) have been considered but are moot in view of the new ground(s) 
of rejection. 

Certain arguments remain relevant to the new grounds of rejections and are 
appropriately discussed below. 

In particular, the applicant argues with respect to claims 1, 5-6, 9, 21, 41-47 and 
51 that "the Examiner repeatedly states that certain claimed limitations are inherent, well- 
known, or predictable (Remarks of 05/08/2009: Page 23, paragraph 03)." The examiner 
would first like to clarify that the only application of any element of the rejection that has 
been determined "predictable" are the results of modifications that have otherwise been 
determined as obvious. 
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The applicant is additionally reminded that proper disclosure of the prior art is not 
limited to what the references explicitly disclose, but also includes what is suggested, 
implied, inherent, or rendered obvious by the disclosures of the prior art to one of 
ordinary skill in the art. 

In particular, obviousness to one of ordinary skill in the art has been re-defined 
by the Supreme Court with the ruling in KSR International Co. v. Teleflex Inc. 
(hereinafter "KSR"). Prior to the ruling in KSR, the standard application to determine 
obviousness was considered to be the "teaching, suggestion, or motivation" test 
(hereinafter "TSM"). The findings of KSR stated that reliance exclusively on TSM had 
previously been flawed in concluding "that a patent claim cannot be proved obvious 
merely by showing that the combination of elements was 'obvious to try' (KSR, Opinion 
of the Court: Page 17, paragraph 02)." The Supreme Court further stated that: 

"When there is a design need or market pressure to solve a problem and there 
are a finite number of identified, predictable solutions, a person of ordinary skill 
has good reason to pursue the known options within his or her technical grasp. If 
this leads to the anticipated success, it is likely the product not of innovation but 
of ordinary skill and common sense (KSR, Opinion of the Court: Page 17, 
paragraph 02)." 

The examiner contends that given this broader application of obviousness the 
teachings of the prior art as cited provide sufficient disclosure and motivation to provide 
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one of ordinary skill in the art incentive to combine the teachings of the prior art in a way 
that would be applicable to the limitations of the claims of the instant invention. It is 
noted by the examiner that the disclosure of the prior art is not limited to the explicit 
teachings of the same, but further includes any teachings that are inherent, implied, or 
suggested such that one of ordinary skill in the art would be aware of them. It is further 
argued by the examiner that the intended usage of the prior art need not match that of 
the instant invention provided that the prior art sufficiently provides teachings and 
motivations to provide one of ordinary skill in the art good reason to pursue options 
within his or her technical grasp. In the previous rejection, the examiner has attempted 
to provide clear and concise evidence and reasoning to show how the limitations in 
question are rendered obvious to try in this fashion by the teachings of the prior art to 
one of ordinary skill in the art. 

The examiner further contends that with particular respect to the teachings of the 
prior art, when such analysis showing obviousness is provided in prosecution the 
burden rests with the applicant in discrediting the assertion that the instant invention is 
rendered obvious by the prior art using objective evidence, reasoning or rationale. It is 
noted by the examiner that such counter-analysis is not provided in sufficient detail to 
illustrate the flaws of the previous rejections. For at least these reasons, the examiner 
finds the applicant's arguments against the previously cited inherency and obviousness 
analyses to be non-persuasive. 

Lastly, the applicant argues with respect to the previous rejection of claims 7, 
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12-13, 27, and 32-33 that the teachings of Nicholson (US PGPUB 2002/0067859; cited 
previously) are "not directed toward any field of endeavor that remotely resembles that of 
Applicants' invention. . . Nicholson does not pertain to any sort of digital videography 
techniques. . . Nicholson is non-analogous art, and is therefore not relevant with respect to 
Applicants' claimed invention [emphasis in original] (Remarks of 05/08/2009: Page 26, 
paragraph 02)." 

The examiner first notes that the claims are not directly specifically and 
exclusively to videography, but explicitly recite a photographic target and 
audio/video data. At minimum, these limitations of the claims would mean that the 
field of relevant art would include photographic data, such as is disclosed by Nicholson. 
However, the examiner further contends that one of ordinary skill in the art of 
videography would additionally be at least familiar with the art of photographic 
processing, as the visual data in the field of videography is composed of a succession 
of photographic images. This classification is particularly relevant in the digital domain, 
to which the teachings of Nicholson are directed. Therefore, the examiner contends 
that the procession of video data is analogous to the processing of photography and 
that one of ordinary skill in the art of videography, as is applicable to the instant 
application, would additionally be familiar with the principles of photography as well. 

The examiner further notes that the previously cited field of art considered for the 
teachings of Nicholson comprises media data storage, search, organization, and 
retrieval. Specifically, the examiner contended that each of the combined references 
Newman (US PGPUB 2003/0101 156; cited previously) and Nicholson are directed to "a 
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system for retrieving media data using labels for the purpose of 
allowing better organization of said data (Office Action of 03/05/2009: 
Page 30, paragraph 05) . " The examiner further contends that the applicant provides 
no further arguments regarding the specific reasons why Nicholson fails to be included 
in this particular field of art. 

For at least the above reasons, the examiner contends that the applicant's 
arguments regarding the teachings of Nicholson as non-analogous art are non- 
persuasive. 



Claim Rejections - 35 USC §112 

8. The following is a quotation of the first paragraph of 35 U.S. C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

9. Claims 49 and 52 rejected under 35 U.S.C. 1 1 2, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. 
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With regard to claim 49, the amended version of the claim comprises the added 
limitations of "said label amplitude parameter being based upon a 
narration amplitude, said label duration parameter being based 
upon a duration of said narration." Neither limitation is properly supported 
by the original disclosure of the application. Therefore each limitation comprises new 
matter. 

With regard to claim 52, this new claim comprises a limitation of "head- 
mounted sound-sensor device that is worn in close proximity to 
said narrator . " Such an arrangement is not properly supported by the original 
disclosure of the application, and therefore said limitation comprises new matter. 

1 0. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

11. Claim 49 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

With respect to claim 49, the terms "label amplitude" and "label 
duration" are not terms well known within the art and are not particularly defined 
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within the disclosure of the instant application. The examiner contends that there is no 
clear meaning for either of these terms that would be understood by one of ordinary skill 
in the art without further clarifications. The previous disclosure of the application would 
mostly likely lead one of ordinary skill in the art to understand a "label amplitude" of a 
text label to be the "amplitude of the text label" and a "label duration" of a text label to 
be the "duration of the text label." 

As noted above, the examiner contends that there is no clear meaning for either 
of these terms, either to reinforce or discourage an appropriate usage of either term. 
The additional limitations of "said label amplitude parameter being based 
upon a narration amplitude, said label duration parameter being 
based upon a duration of said narration" appear to attempt to redefine this 
interpretation for one of ordinary skill in the art, but such definitions are not present in 
view of the specification of the instant application. For this reason, the limitations of 
claim 49 comprise ambiguous subject matter which does not particularly point out and 
distinguish what is considered the applicant's invention from the prior art. 



Claim Rejections - 35 USC § 103 



12. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 
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13. Claims 1-2, 4-6, 8-10, 15-17, 21-22, 24, 28, 30, 35-37, 41-47 and 50-51 are 

rejected under 35 U.S.C. 103(a) as being unpatentable over Newman in view of 
Kanevsky (US Patent 6,434,520; previously cited but not relied upon). 

Regarding claim 1, Newman discloses a system for cataloguing information 
comprising: 

• an electronic device that captures audio/video data 
corresponding to a photographic target [data acquisition device] 
(Col. 2, paragraphs 0012-0014; Col. 3, paragraph 0032), 

• said audio/video data [Audio, photo and/or video; APV data] including 
a narration [audio messages] concurrently provided by a 
narrator specifically to mark where respective subject 
matter locations are positioned in said audio/video data 
[selectively retrieved] (Col. 2, paragraphs 0019, 0022-0023, 0025; Col. 3, 
paragraphs 0032-0033); 

• a speech recognition engine that automatically performs a 
speech recognition process upon said narration to generate 
labels [tags] that correspond to said respective subject 
matter locations in said audio/video data (Page 2, paragraph 
0017; Page 3, paragraphs 0025, 0033), 



Application/Control Number: 10/805,781 Page 14 

Art Unit: 2626 

• said labels being text conversions of utterances in said 
narration [converting audio files to text files] (Page 2, paragraph 0017; Page 
3, paragraph 0025; Page 3, paragraph 033 - Page 4, paragraph 0036), 

• said labels each being specifically aligned with 
corresponding ones of said respective subject matter 
locations within said audio/video data [storing message with data; 
logs with indicators; tags for identification data] (Page 2, paragraphs 0013, 0017, 
0019; Page 3, paragraph 0033); 

Kanevsky additionally discloses a system for indexing audio/video data 
[audio/multimedia files] (Abstract) comprising: 

• a speech recognition engine that automatically 
performs a speech recognition process upon said 
narration to generate labels that correspond to said 
respective subject matter locations in said 
audio/video data [regularly extracting audio data stream into 
segments; speech recognition/transcription module decodes the spoken 
utterances in segments] (Fig. 1, elements 102, 109; Col. 3, lines 12-19; 
Col. 6, lines 39-44), 

• said labels being text conversions of utterances in 
said narration [decoding spoken utterances and generating a 
corresponding transcription] (Col. 6, lines 39-44), 
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• said labels each being specifically aligned with 
corresponding ones of said respective subject matter 
locations within said audio/video data [processing feature 
vectors by determining the locations of changes in speaker, channel 
and/or background and then marking said locations] (Fig. 1, element 103; 
Col. 3, lines 19-29); and 

• a label manager that manages a label mode for 
generating and storing said labels [audio indexing system 
with database], said label manager also controlling a 
label search mode that utilizes said labels to 
automatically locate said respective subject matter 
locations in said audio/video data [database includes tags] 
(Fig. 1, elements 105-111; Col. 7, lines 21-26; Col. 8, lines 21-36, lines 49- 
52). 

The references are combinable because each is directed to the field of media 
storage, organization, and retrieval. Kanevsky further provides motivation to combine 
the references in disclosing the utility of labeling audio [tagging data] based upon 
identity of speakers or other specified circumstances for the purpose of increasing 
efficiency in searching a large volume of media data (Co. 1, lines 28-39). 

Therefore, the examiner contends that it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to combine the teachings of 
Newman and Kanevsky for the purpose of implementing an apparatus for the storage, 
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organization, and retrieval of media data that is further operable to utilize audio labels 
based upon speaker identity or other specified circumstances for the purpose of 
increasing efficiency in searching large volumes of media data. 

Regarding claim 2, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further discloses that said electronic 
device is implemented as an audio/video camcorder device [video 
camera] (Page 2, paragraphs 0014, 0021; Page 3, paragraph 0032). 

Regarding claim 4, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further discloses that said label manager 
initially instructs said electronic device to enter a real-time 
label mode [data acquisition devices read at a fixed time interval] for creating 
and storing said labels [tags] (Page 3, paragraphs 0032-0033), said 
electronic device concurrently [run continuously or run selectively at desired 
times] capturing said audio/video data and said narration after 
said label manager instructs said electronic device to enter 
said real-time label mode (Page 3, paragraph 0032) . 

Regarding claim 5, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further renders obvious that said 
electronic device enters a real-time label mode in response to a 
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verbal label-mode command from a system user, said verbal label- 
mode command being recognized and provided to said label manager 
by said speech recognition engine in disclosing a system for labeling [tags] 
data (Page 3, paragraph 0033) that is responsive to selective searching (Page 2, 
paragraph 0022 - Page 3, paragraph 0023), and additionally includes voice recognition 
components for entering data (Page 3, paragraph 0025). 

Because Newman teaches both verbal commands of the system, and also 
teaches labeling data, both elements of the claim are known in view of the teachings of 
Newman. Furthermore, speech control is one of a finite number of input methods for 
the system as taught by Newman and there is reasonable expectation of success for 
one of ordinary skill in the art to adapt the speech control disclosure of Newman to 
accommodate input of label [tag] data. The examiner further contends that any system 
that operates on the time-dependent basis of voice-input must inherently operate in 
real-time for proper functional operation to the end-user. Therefore, because Newman 
teaches all components of the limitations of claim 5, including a verbal command mode 
as one of a finite number of inputs to the system and an input of label data mode, and 
one of ordinary skill in the art would have no reason not to expect success in adapting 
the verbal commands to include input of label data, it would have been obvious for one 
of ordinary skill in the art at the time the invention was made to modify the teachings of 
Newman to accommodate greater flexibility of data input by doing so. 



Regarding claim 6, Newman in view of Kanevsky discloses all limitations of 
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claim 1 as applied above, and Kanevsky further discloses that said speech 
recognition engine automatically generates said labels as said 
electronic device captures said audio/video data and said 
narration [source of audio stream may be conversational speakers; unsupervised 
segmentation] (Fig. 2A; Fig. 2B; Col. 2, lines 63-66; Col. 3, lines 55-59) 

Regarding claim 8, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further discloses that said label manager 
stores said labels during a real-time label mode, said labels 
being stored along with meta-inf ormation that associates each of 
said respective subject matter locations to a corresponding one 
of said labels [date and time] (Page 3, paragraph 0033). 

Regarding claim 9, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Kanevsky further discloses that said electronic 
device initially captures said audio/video data and said 
narration prior to entering said label mode [audio Stream may be a pre- 
recorded video with accompanying audio track] (Col. 2, lines 63-66). 

Regarding claim 10, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further discloses that said label manager 
instructs said electronic device to enter a non-real-time label 
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mode [data access after operation is complete] for creating and storing said 
labels [data review], said electronic device responsively retrieving 
and playing back said audio/video data and said narration (Page 4, 
paragraph 0035-0037) . 

Regarding claim 15, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further discloses that said label manager 
stores said labels in a non-real-time label mode, said labels 
being stored along with meta-inf ormation that associates each of 
said respective subject matter locations to a corresponding one 
of said labels [tags] (Page 3, paragraph 0032-0033; Page 4, paragraph 0035- 
0037). 

Regarding claim 16, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Kanevsky further discloses a system user 
interactively selects a search label for performing a label 
search procedure to locate a specific one of said respective 
subject matter locations corresponding to said search label [user 
may retrieve stored audio segments from the database by formulating queries based on 
one or more parameters corresponding to indexed information] (Col. 1, line 61 - Col. 2, 
line 04). 
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This limitation is directly related to the teachings of Kanevsky applied above to 
the limitations of claim 1 . Therefore, the motivation to combine the references is the 
same for claim 16 as applied above to claim 1. 

Regarding claim 17, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further implies that said label manager 
generates a label-search GUI on a display of said electronic 
device, a system user viewing said labels and corresponding 
representative images [icons] from said audio/video data for 
selecting a search label (Page 4, paragraph 0035-0038) in disclosing that the 
data retrieval system as disclosed by Newman includes a selective display capability for 
presenting relevant data to a user. 

Regarding claims 21-22, 24-26, 28-30, and 35-37, these claims are very similar 
to claims 1-2, 4-6, 8-10, and 15-17 respectively as applied above, and are rejected for 
the same reasons. 

Regarding claims 41-42 and 44-47, each of these claims is very similar to claim 
1 as applied to above, and is rejected for the same reasons. 

Regarding claim 43, this claim contains limitations very similar to those found in 
a combination of claims 1, 8, 13, and 15, which are all addressed by Newman 
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individually, and therefore claim 43 is rejected for the same reasons. 

Regarding claim 50, Newman in view of Kanevsky discloses all limitations of 
claim 17 as applied above, and Newman further discloses that said 
representative images are implemented as thumbnail images [icons 
corresponding to an image related to data] (Page 4, paragraphs 0037, 0039). 

Regarding claim 51, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Newman further renders obvious that said 
electronic device is a single discrete video camcorder that 
hosts said speech recognition engine, said label manager, said 
labels, and said audio/video data (Page 1 , paragraph 0006; Page 2, 
paragraphs 0014-0015, 0021 ; Page 3, paragraph 0032) in disclosing the inclusion of a 
camcorder as a device for initial data capture [video camera] (Page 2, paragraph 0014; 
Page 3, paragraph 0033), and that the system can be embodied in a single device 
(Page 1, paragraph 0006). 

Because such limitations of the claim are taught by Newman, they would have 
been known to one of ordinary skill in the art in view of Newman at the time the 
invention was made. Furthermore, motivation to combine the elements taught in 
Newman would exist because it is well-known in the art to integrate functionality of 
system into a single portable device to increase convenience and transportability of said 
system. 
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The references are combinable because each is directed to the field of media 
storage, organization, and retrieval. Kanevsky further provides motivation to combine 
the references in disclosing the utility of labeling audio [tagging data] based upon 
identity of speakers or other specified circumstances for the purpose of increasing 
efficiency in searching a large volume of media data (Co. 1 , lines 28-39). 

Therefore, the examiner contends that it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to combine the teachings of 
Newman and Kanevsky for the purpose of implementing an apparatus for the storage, 
organization, and retrieval of media data that is further operable to utilize audio labels 
based upon speaker identity or other specified circumstances for the purpose of 
increasing efficiency in searching large volumes of media data and is further 
implemented on a single device encapsulating all elements of the system for the 
purpose of increasing convenience and transportability of said system. 

14. Claims 3, 14, 18-20, 23, 34, and 38-40 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Newman in view of Kanevsky and in further view of Belrose 
(US Patent Application Publication2003/01 44843), cited in a prior Office Action. 

Regarding claim 3, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above. 

Belrose additionally discloses: 
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• said speech recognition engine is configured in a 
simplified configuration [specific queries] that efficiently 
compares said narration with acoustic models to 
identify phone strings [recognizing a user information request] 
that represent said narration [speech input] (Page 2, 
paragraph 0030; Page 3, paragraphs 0047, 0051-0052), 

• said speech recognition engine referencing a compact 
dictionary to look up recognized vocabulary words that 
correspond to said phone strings [one or more queries] (Page 
3, paragraph 0048), 

• said speech recognition engine utilizing a limited set 
of recognition grammar to form said recognized 
vocabulary words into said labels [label for a feature] that 
are supported by said speech recognition engine (Page 6, 
paragraph 0106). 

The references are combinable because each is directed to a system for 
capturing, storing, and browsing audio and/or video data and organizing said data using 
labels for ease of access to said data. Kanevsky further provides motivation to combine 
the references in disclosing the utility of labeling audio [tagging data] based upon 
identity of speakers or other specified circumstances for the purpose of increasing 
efficiency in searching a large volume of media data (Co. 1 , lines 28-39). Belrose 
further provides motivation to combine the references in disclosing the utility of 
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encapsulating such a system into a mobile device [cellular telephone] operable using 
voice commands (Page 1 , paragraph 0003) to allow free mobility of the system. 

Therefore, the examiner contends that it would have been obvious to combine 
the teachings of Newman with the teachings of Kanevsky and Belrose in order to 
implement a media browser organized using labels that is operable to utilize audio 
labels based upon speaker identity or other specified circumstances for the purpose of 
increasing efficiency when searching large volumes of audio, and is further 
encapsulated into a mobile device operable using voice commands to allow free 
mobility of the system. 

Regarding claim 14, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above. Belrose additionally discloses that said label manager 

coordinates a label validation procedure for validating said 
labels in response to verbal validation commands from a system 
user [identification], said verbal validation commands being 
recognized and provided to said label manager by said speech 
recognition engine [determine the nature of the information to be recorded] (Page 
6, paragraph 0098-0108). 

This limitation is directly related to the voice command input limitation disclosed 
by Belrose as applied above to claim 3. Therefore, the motivation to combine the 
references is the same for claim 14 as applied above for claim 3. 
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Regarding claim 18, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above. Belrose additionally discloses that a system user 
selects a search label by issuing a verbal search-label command 
[hotspot label], said verbal search-label command being recognized 
and provided to said label manager by said speech recognition 
engine (Page 6, paragraph 01 06; Page 7, paragraphs 01 1 0-01 1 3) . 

This limitation is directly related to the voice command input limitation disclosed 
by Belrose as applied above to claim 3. Therefore, the motivation to combine the 
references is the same for claim 18 as applied above for claim 3. 

Regarding claim 19, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above. Belrose additionally discloses that said label manager 

instructs said electronic device to automatically locate and 
retrieve a specific one of said respective subject matter 
locations in response to a system user selecting a search label 
[hotspot dialogue blocks] (Page 6, paragraphs 0106-0108). 

This limitation is directly related to the voice command input limitation disclosed 
by Belrose as applied above to claim 3. Therefore, the motivation to combine the 
references is the same for claim 19 as applied above for claim 3. 

Regarding claim 20, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above. Belrose additionally discloses that said electronic 
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device automatically plays back a specific retrieved one of said 
respective subject matter locations [greeting dialog block specific to 
narrator] from said audio/video data for viewing by said system 
user (Page 6, paragraphs 0097-0100). 

This limitation is directly related to the voice command input limitation disclosed 
by Belrose as applied above to claim 3. Therefore, the motivation to combine the 
references is the same for claim 20 as applied above for claim 3. 

Regarding claim 23, this claim is very similar to claim 3 and is rejected for the 
same reasons. 

Regarding claim 34, this claim is very similar to claim 14 and is rejected for the 
same reasons. 

Regarding claims 38-40, these claims are very similar to claims 18-20 

respectively and are rejected for the same reasons. 

1 5. Claims 7, 12-13, 27, and 32-33 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Newman in view of Kanevsky and in further view of Nicholson (US 
Patent Publication Application 2002/0067859), cited in the previous Office Action. 
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Regarding claim 7, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above. Nicholson additionally discloses a post processor [digital 
processor] operating in real-time to perform a validation procedure for the labels based 
upon confidence measures [threshold confidence level] (Page 2, paragraph 0012). 

The references are combinable because each is directed to a system for 
retrieving media data using labels for the purpose of allowing better organization of said 
data. Kanevsky further provides motivation to combine the references in disclosing the 
utility of labeling audio [tagging data] based upon identity of speakers or other specified 
circumstances for the purpose of increasing efficiency in searching a large volume of 
media data (Co. 1 , lines 28-39). Nicholson further provides motivation to combine in 
disclosing the utility of a system that is able to discriminate between identifiable and 
non-identifiable data for the purpose of maintaining a high level of recognition by the 
user of decoded data with a minimal additional storage cost (Page 2, paragraph 0014). 

Therefore, the examiner contends that it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to combine the teachings of 
Newman and Nicholson in order to implement a system for retrieving media data that is 
operable to utilize audio labels based upon identity of speakers or other specified 
circumstances for the purpose of increasing efficiency in searching a large volume of 
media data, and further is operable to use labels that utilizes discrimination between 
identifiable and non-identifiable data in order to maintain a high level of user recognition 
of decoded data with minimal additional storage costs. 
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Regarding claim 12, this claim is very similar to claim 7 and is rejected for the 
same reasons. 

Regarding claim 13, Newman in view of Kanevsky discloses all limitations of 
claim 1 and additionally discloses a validation graphical user interface [display] allowing 
users to evaluate, delete, and edit labels [tags] (Page 4, paragraphs 0035-0037). 

Nicholson additionally discloses coordinating a label validation procedure for 
validating said labels (Page 2, paragraph0012). 

The limitations of claim 13 are very similar to those of claim 7, and therefore the 
motivation to combine the references is the same for claim 13 as for claim 7. 

Regarding claim 27, this claim is very similar to claim 7 and is rejected for the 
same reasons. 

Regarding claims 32-33, these claims are very similar to claims 12-13 

respectively, and each is rejected for the same reasons respectively. 

16. Claims 1 1 , 31 , and 48 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Newman in view of Kanevsky in further view of Adams (US Patent 
Application Publication 2004/0008209), cited in a previous Office Action. 
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Regarding claim 11, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above. Adams additionally discloses automatically [addressing 
devices automatically] generating labels [meta-data] by analyzing audio/video data and 
narration data during playback of said audio/video data and said narration data (Page 3, 
paragraph 0074-0080). 

The references are combinable because each is directed to a system for storage 
and retrieval of media data. Kanevsky further provides motivation to combine the 
references in disclosing the utility of labeling audio [tagging data] based upon identity of 
speakers or other specified circumstances for the purpose of increasing efficiency in 
searching a large volume of media data (Co. 1 , lines 28-39). Adams provides 
motivation in disclosing the utility of providing automated association of meta-data to 
audio/video data in order to allow for more efficient organization of audio/video data 
(Page 6, paragraph 01 1 1 ). 

Therefore, the examiner contends that it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to combine the teachings of 
Newman, Kanevsky and Adams in order to implement a system for storage and retrieval 
of media data that utilizes audio labels based upon identity of speakers or other 
specified circumstances for the purpose of increasing efficiency in searching a large 
volume of media data and is further operable to provide automated association of meta- 
data to said media data in order to allow for more efficient organization of said media 
data. 
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Regarding claim 31 , this claim is very similar to claim 1 1 and is rejected for the 
same reasons. 

Regarding claim 48, Newman in view of Kanevsky discloses all limitations of 
claim 8 as applied above. Adams additionally implies the use of video timecode 
information as a component of said meta-data in disclosing the storage of meta-data 
that comprises arrangement data with regard to media data such as video data (Page 3, 
paragraph 0079), because timecode information is a well-known and readily-available 
method of arranging frames of video data. 

The motivation to combine the references as applied to claim 48 is the same as 
applied above to claim 11, because each claim presents limitations that are applicable 
to the automatic arrangement and organization of data and meta-data on a storage 
component of a media data retrieval system. 

17. Claim 52 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Newman in view of Kanevsky and in further view of US Patent 6,538,623, hereinafter 
referred to as Parnian. 

Regarding claim 52, Newman in view of Kanevsky discloses all limitations of 
claim 1 as applied above, and Kanevsky further discloses a Narration being 

identified for conversion in to said labels by having a greater 
amplitude than other ambient sound that is recorded from more 
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remote sources as part of said audio/video data [determining acoustic 
components of speech using signals such as speech in noisy environments] (Col. 2, line 
63 - Col. 3, line 1 1 ; Col. 3, lines 40-52). 

Parnian additionally discloses a head-mounted sound-sensor device 
that is worn in close proximity to said narrator in disclosing a mobile 
device that provides a plurality of media-gathering tools including a wearable 
microphone (Abstract; Fig. 6A, element 35c; Col. 4, lines 49-60; Col. 10, lines 26-29, 
lines 47-64). 

The references are combinable because each is directed a system of media 
storage, organization, and retrieval. Kanevsky further provides motivation to combine 
the references in disclosing the utility of labeling audio [tagging data] based upon 
identity of speakers or other specified circumstances for the purpose of increasing 
efficiency in searching a large volume of media data (Co. 1 , lines 28-39). Parnian 
further provides motivation to combine the references in disclosing the utility of a mobile 
data collection and retrieval device that allows for mobile collection of media data with 
timestamps to maintain integrity of the data for the purpose of allowing a user to make 
use of the device to capture data in a remote location (Col. 4, lines 49-60; Col. 10, lines 
55-64). 

Therefore, the examiner contends that it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to combine the teachings of 
Newman, Kanevsky, and Parnian in order to implement a system of media storage, 
organization, and retrieval that is further operable to utilize audio labels based upon 
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identity of speakers or other specified circumstances for the purpose of increasing 
efficiency in searching a large volume of media data, and is further implemented as a 
mobile data collection and retrieval device that allows for mobile collection of media 
data with timestamps maintain integrity of the data for the purpose of allowing a user to 
make use of the device to capture data in a remote location. 



Conclusion 

18. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Bates (US Patent 4,272, 790) teaches a video tape editing system that 
includes synchronization data between multiple tapes using timecode. 

• Ardis (US Patent 5,1 72,281 ) teaches a video transcript retriever that 
includes usage of timecode. 

• Orphan (US Patent 5,508,754) teaches a system for encoding and 
displaying captions for television programs using timecode for 
synchronization. 

• Ludwig (US Patent 5,617,539) teaches a multimedia collaboration system 
with separate data networks and A/V network controlled information 
transmission. 

• Hill (US Patent 5,636,283) teaches a system for mixing audio, including 



Application/Control Number: 10/805,781 Page 33 

Art Unit: 2626 

usage of timecode for synchronization of audio and video. 

• Renie (US Patent 5,655,053) teaches a personal video capture system 
including a video camera at a plurality of different locations, including 
synchronization features. 

• Peters (US Patent 5,905,841 ) teaches an electronic film editing system 
using both film and videotape formats, including synchronization features. 

• Nunally (US Patent 5,917,958) teaches a distributed video data base with 
remote searching for image data features. 

• Menard (US Patent 6,061 ,056) teaches a television monitoring system 
with automatic selection of program material of interest and subsequent 
display under user control. 

• MacCormack (US Patent 6,144,797) teaches an intelligent video 
information management system performing multiple functions in parallel. 

• Foote (US Patent 6,404,925) teaches an apparatus for segmenting and 
audio-visual recording using image similarity searching and speaker 
recognition. 

• Tritschler (US Patent 6,424,946) teaches an apparatus for unknown 
speaker labeling using concurrent speech recognition, segmentation, 
classification and clustering. 

• Swaminathan (US Patent 6,425,525) teaches a system and method for 
inputting, retrieving, organizing, and analyzing data. 
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• Jones (US PGPUB 2002/01 88841 ) teaches a digital asset management 
and linking media signals with related data using watermarks. 

• Basu (US PGPUB 2003/0018475) teaches a method and apparatus for 
audio-visual speech detection and recognition. 

• Barber (US PGPUB 2003/01 6531 9) teaches a multimedia recording 
system including organization and cataloguing features. 

• Fernandez (US PGPUB 2004/0260669) teaches a network-extensible 
reconfigurable media appliance. 

• Divakaran (US PGPUB 2005/0125223) teaches an audio-visual highlight 
detection using coupled Hidden Markov Models. 

• Bolle (US Patent 6,993,535) teaches a method and apparatus for inducing 
multimedia classifiers based on features of disparate modalities. 

• Reynar (US Patent 7,003,522) teaches a system and method for 
incorporating smart tags in online content. 

• Abbott (US Patent 7,155,456) teaches a system for storing and recalling 
information of different types using tags, including video, audio, and 
textual data. 

• Danner (US Patent 7,219,136) teaches an apparatus and method for 
providing network-based information suitable for audio output. 

• Julia (US Patent 7,220,073) teaches a system and method for speech 
activated navigation. 
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• Colbath (US Patent 7,290,207) teaches a system and method for 
providing multimedia information management. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David Kovacek whose telephone number is (571)270- 
3135. The examiner can normally be reached on M-F 9:00am - 5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Hudspeth can be reached on (571) 272-7843. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



DMK, 07/20/2009 



/Talivaldis Ivars Smite/ 
Primary Examiner, Art Unit 2626 



7/20/2009 



Application/Control Number: 10/805,781 Page 36 

Art Unit: 2626 



